In general, in older and current video coding standards and recommendations, a Group of Pictures (GOP) structure only involves GOP length (N) and picture type (i.e., P-picture interval M) selection. Such older video coding standards and recommendations include, for example, the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-1 (MPEG-1) standard, the ISO/IEC MPEG-2 standard, the International Telecommunication Union, Telecommunication Sector (ITU-T) H.263 recommendation. A new video compression standard/recommendation, the ISO/IEC MPEG-4 Part 10 Advanced Video Coding (AVC) standard/ITU-T H.264 recommendation (hereinafter the “MPEG-4 AVC standard”), provides several new tools to improve coding efficiency.
Similar to older video coding standards and recommendations, the MPEG-4 AVC standard uses/supports three different picture (slice) types (I, P and B pictures (slices)). Moreover, the MPEG-4 AVC standard includes new tools/features to improve coding efficiency.
For example, the MPEG-4 AVC standard decouples the order of reference pictures from the display order. In prior video coding standards and recommendations, there was a strict dependency between the ordering of pictures from motion compensation purposes and the ordering of pictures for display purposes. In the MPEG-4 AVC standard, these restrictions are largely removed, allowing the encoder to choose the reference order and display order with more flexibility.
Moreover, the MPEG-4 AVC standard decouples picture representation methods from picture referencing capability. In prior video coding standards and recommendations, B pictures cannot be used as references for the prediction of other pictures in the video sequence. In the MPEG-4 AVC standard, there is no such constraint. Any picture type can be used as a reference picture or a non-reference picture.
Further, the MPEG-4 AVC standard allows multiple reference pictures for motion compensation. With these new features, when a GOP structure is selected, we need to consider not only the GOP length and the picture type selection, but also the coding order of the picture and the reference picture selection.
Most previous work related to the GOP structure has been concentrated on GOP length and picture type selection. The GOP length is, in general, fixed by the application. When dynamic GOP length is allowed, the first picture after the scene change is coded as an I-picture, and the next GOP is merged to the current GOP.
In a first prior art approach, a method is disclosed in which the GOP structure is adapted by taking into account temporal segmentation. That is, picture types are adjusted according to the temporal variation of the input video.
In a second prior art approach, it is disclosed that the optimal picture type in the GOP may be selected from possible candidates by solving a minimization problem with the Lagrange multiplier method.
In a third prior art approach, a system is disclosed wherein macroblock activity information is used to decide picture type.
As noted above, most of the prior art related to the GOP structure has only concentrated on GOP length and picture type selection. However, the consideration of only GOP length and picture type selection disadvantageously limits the flexibility of the MPEG-4 AVC standard.